Conventionally, in a data storage environment the data management tools used to load and transform large volumes of data for a target database are complicated and require certain expertise. This in turn causes delays when large data volumes are expected to be migrated to a database server or comparable network device.
Most data management systems currently available on the market today perform a three-part operation including the extraction, transformation and loading (ETL) of data. The ETL tools design a user interface that permits the user to create a process map to copy data from a source, transform that data, and then load the data to a destination. The map is saved and is loaded on a server that runs the map and performs processes included on the map. Users have to become experts in the ETL user interface to accomplish medium to complex data transformations and as a result each ETL system has a different interface.
Conventional data migration applications may move source data twice, such as in ETL tools like SSIS, which can be costly when loading large amounts of data. In addition, in ETL tools the data has to be moved from the destination database to the executing server in order to compare it to the source data in memory so that proper data transformations can be made. This results in large amounts of data being moved from server to server.